1. Field of the Invention
This invention relates to methodology and system for recapturing a trajectory of an object.
2. Introduction to the Invention
Tracking objects is a common measurement task in situations as different as keeping space probes in a correct orbit, or watching packaged goods go by in a wrapping plant.
There are known tracking systems and methods to do this locally, i.e., with a single sensor such as a video camera/feature extraction system. However, it is significantly harder to extend this methodology across multiple sensors, because of the alignment required. That is, each sensor may do an excellent job of measuring the position and velocity of an object with respect to that sensor""s own coordinate system, but relating that coordinate system to those of all the other sensors and to the ordinary Euclidean 4-space of the real world, is a difficult task in general.
There are three basic strategies for handling this task.
The first strategy is to make a single, highly capable sensor that can do all the work itself, e.g., an extremely high resolution IR imaging camera and machine vision processor, or a big Doppler radar. The second strategy is to carefully align several sensors to a known position and orientation, usually using triangulation. The third strategy is to relate each sensor""s coordinates to the Euclidean space by calibration, and use a coordinate transformation on each sensor""s data individually.
As to this third strategy in particular, we note that calibration and alignment steps are necessary before the data from the sensors can be combined to form an overall measured trajectory for an object. This requirement also places a severe requirement on our knowledge of the geometric imaging properties of the sensors; if the sensor is not a nice regular array of pixels, there is no one combination of rotation and translation that can map one sensor""s coordinates into another""s. We thus need in principle to know the position of each pixel individually, which is a very stiff constraint, especially since the accuracy requirements of the sensor system may be modest.
There are many situations in which it would be advantageous to set up a sensor array quickly, without a lot of attention paid to exact alignment and aiming, so that one is able to gather good trajectory information.
This invention discloses methodology suitable for building self-configuring networks of multi-pixel sensors, whose geometric patterns are reasonably well known, but whose positions and orientations may be initially completely unknown except for a constraint of overlapping fields of view.
Looking at this situation in another way, the very fact that objects are moving in a measurement space can give us a way of determining relative coordinate transformations between the coordinate systems of different sensors, provided that the objects cross from one sensor""s field of view (FOV) into another""s and there is no significant gap between them, at least along the object""s trajectories. A significant gap is one large enough for an object""s velocity to change sufficiently to make an extrapolation of its position from the time it left one sensor""s FOV to the time it enters another, too inaccurate to use. This obviously depends a great deal on the properties of the object and its motion; the maximum acceleration of a supertanker is on the order of 104 cmsxe2x88x922; a person walking perhaps 50 cmsxe2x88x922; and, an object in a chute, 104 cmsxe2x88x922.
Objects whose trajectories cross an overlap region from one FOV into another provide a geometric proof that the corresponding pixels are looking at the same place in Euclidean space, even if it is not known just where that place is, and this can be used to determine the conformation of the sensor FOVs. In order to discuss this clearly, we refer to the partial trajectory data from a single sensor as a sub-trajectory, the final joined-together product in the system""s measurement coordinate system as a trajectory sequence.
Provided that the data are synchronized in time to a sufficiently fine accuracy and granularity, and that there are not too many objects in the measurement space, the timing of the crossing, and perhaps comparison of the signatures of the objects in each sub-trajectory, can be used to uniquely assign sub-trajectories to the correct sequence. (By a signature, we mean a constant or slowly varying characteristic of data corresponding to an object that allows it to be distinguished from other objects.) In this way, a group of low data rate, low resolution position sensors can be configured automatically into a system that can track objects globally inside a space.
Accordingly, in a first aspect, the present invention discloses a method for recapturing a trajectory of an object, the method comprising the steps of:
1) employing first and second sensors, respectively capable of gathering information about an object""s trajectory within first and second at least partially overlapping fields of view, thereby obtaining a sub-trajectory in each field of view; and
2) stitching the sub-trajectories into a single estimated trajectory.
Step 1) of the method preferably comprises employing first and second multi-pixel infrared sensors.
Preferably, each step 1) sub-trajectory is time stamped, and preferably a time stamp is used for identifying sub-trajectories corresponding to the same object. The method may further comprise a step of using the time stamps for rejecting sub-trajectories which if stitched together would give rise to a discontinuous estimated trajectory.
Step 2) may comprise choosing among sub-trajectories coordinates for stitching based on the sub-trajectories"" spatial proximity in the overlap region.
The method may comprise a situation wherein each sub-trajectory is time stamped, and step 2) comprises choosing among sub-trajectories candidates for stitching based on the sub-trajectories"" spatial proximity in the overlap region. In this regard, the method may comprise a further step of using the time stamps for rejecting stitching candidates which if stitched together would give rise to a discontinuous estimated trajectory.
The method advantageously further comprises a step of employing a third sensor capable of gathering information about an objects"" trajectory within a third field of view overlapping both the first and second fields of view, in which the mutual consistency of two estimated spatial relationships between sensors is improved by adjusting the estimates to reduce the error residuals from pairwise comparison of a trajectory in the overlap regions.
In a second aspect of the present invention, we disclose a method for recapturing a trajectory of an object, the method comprising the steps of:
1) employing first and second sensors, respectively capable of gathering information about an object""s trajectory in first and second fields of view, thereby obtaining a subtrajectory in each field of view, and so disposed that a known characteristic of the object""s motion places bounds on an error committed in extrapolating its subtrajectories between the edges of the fields of view; and
2) stitching the subtrajectories into a single estimated trajectory.
For the purpose of the second aspect of the invention, techniques for extrapolating include e.g., linear extrapolation along an estimated velocity vector (in Euclidean space) with which the object left one field of view, with error bounds obtained from the known characteristic that the object""s acceleration between leaving one field of view and entering the other must be between predetermined bounds.
In a third aspect of the present invention, we disclose a system suitable for recapturing a trajectory of an object, the system comprising:
1) first and second sensors, respectively capable of gathering information about an object""s trajectory within first and second at least partially overlapping fields of view, for obtaining a sub-trajectory in each field of view; and
2) means for stitching the sub-trajectories into a single estimated trajectory.
At least one of the first and second sensors preferably comprises a multi-pixel infrared sensor.
In a fourth aspect of the present invention, we disclose a method for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors, whose fields of view are known to overlap at least partially, the method comprising the steps of:
1) gathering sub-trajectory data for an object moving in the fields of view;
2) selecting from the sub-trajectories those which correspond to the object; and
3) computing the spatial relationship from the offsets between the selected sub-trajectories.
It is helpful if each sensor has at most one moving object in its field of view. Preferably, an object has a recognizable signature that can differentiate it from other objects.
Preferably, step 2) includes choosing among only those sub-trajectories whose time extents overlap.
Preferably, step 3) comprises applying a condition that a trajectory of an object is continuous in space and time.
The method advantageously further comprises incorporating a third multi-pixel infrared sensor, in which the mutual consistency of two spatial relationships between sensors may be improved by the imposition of a priori geometrical constraints.
In a fifth aspect of the present invention, we disclose a method for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors, the method comprising the steps of:
1) gathering sub-trajectory data for an object moving in the fields of view of the sensors, where a known characteristic of the object""s motion places bounds on an error committed in extrapolating its subtrajectories between the edges of the fields of view;
2) selecting from the sub-trajectories those which correspond to the object;
3) extrapolating the subtrajectories until they overlap in time; and
4) computing the spatial relationship from the offsets between the selected sub-trajectories.
It is helpful if each sensor has at most one moving object in its field of view. Preferably, an object has a recognizable signature that can differentiate it from other objects.
For the purpose of the fifth aspect of the invention, techniques for extrapolating include e.g., linear extrapolation along an estimated velocity vector (in Euclidean space) with which the object left one field of view, with error bounds obtained from the known characteristic that the object""s acceleration between leaving one field of view and entering the other must be between predetermined bounds.
In a sixth aspect of the present invention, we disclose a system suitable for determining a spatial relationship between the coordinate systems corresponding to two multi-pixel sensors whose fields of view are known to overlap at least partially, the system comprising:
1) means for gathering sub-trajectory data for an object moving in the fields of view;
2) means for selecting from the sub-trajectories those which correspond to the object; and
3) means for computing the spatial relationship from the offsets between the selected sub-trajectories.